Adaptive Sampling and Fast Low-Rank Matrix Approximation

We prove that any real matrix A contains a subset of at most 4k/ + 2k log(k + 1) rows whose span “contains” a matrix of rank at most k with error only (1 + ) times the error of the best rank-k approximation of A. We complement it with an almost matching lower bound by constructing matrices where the span of any k/2 rows does not “contain” a relative (1 + )-approximation of rank k. Our existence result leads to an algorithm that finds such rank-k approximation in time O M k + k log k + (m+ n) k 2 + k log k + k log k , i.e., essentially O(Mk/ ), where M is the number of nonzero entries of A. The algorithm maintains sparsity, and in the streaming model [12, 14, 15], it can be implemented using only 2(k+ 1)(log(k+ 1)+ 1) passes over the input matrix and O min{m,n}( k + k log k) additional space. Previous algorithms for low-rank approximation use only one or two passes but obtain an additive approximation.